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(54) Method and apparatus for monitoring circumference of vehicle 



(57) A vehicle's drcunrtference monitoring method is 
provided together with an apparatus adopting the 
method. In the apparatus, it is judged whether a picture 
taken by a CCD camera 13 is inappropriate for monitor- 
ing the vehicle's circumference. When it is judged that 
the picture is improper, it is regarded in the apparatus 



that the monitoring based on the improper picture has a 
reduced reliability in monitoring results. Then, the appa- 
ratus informs a passenger on the vehicle of a fact that 
the monitoring accuracy Is deteriorated. 
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Description 

BACKGROUND OF THE INVENTION 

1. Field of the Invention 

[0001] The present invention relates to a vehicle's 
circumference monitoring method which is employed to 
determine whether or not there exists an obstacle (or 
obstacles) around a vehicle owing to the application of a 
designated image processing on the circumferential pic- 
ture taken by using an imaging unit installed in the vehi- 
cle, for example, a CCD camera. Particularly, it relates 
to a vehicle's circumference monitoring method which 
can provide the result of monitoring the circumference 
of the owner's vehicle with high-leveled reliability and 
also related to a vehicle's circumference monitoring 
apparatus adopting the vehicle's circumference nfx>ni- 
toring method. 

2. Description of the Related Art 

[0002] Japanese Patent Application Laid-open Pub- 
lication No. 7-5076 9 discloses a vehicle's circumfer- 
ence monitoring method which may be used when it is 
required to monitor the existence of the obstacle (or 
obstacles) around the vehicle. According to the monitor- 
ing method, it has been carried out to apply the desig- 
nated image processing on the picture around the 
vehicle brought by the imaging unit, for example, the 
CCD camera. 

[0003] In detail, it is executed to take "^rear-side" pic- 
tures in sequence through the traveling vehicle. Upon 
selection of two sequence frames (scenes) from so- 
taken consecutive "rear-side" pictures, it is further exe- 
cuted to detect a displacement of an identical point 
through the sequence frames over as "optical flow". On 
the basis of the so-detected optical fbw, there can be 
monitored the accessibility of a following vehicle against 
the driver's own vehicle or the same of another vehicle 
traveling on the neighboring traffic lane against the 
driver's own vehicle. 

[0004] According to the above-mentioned prior art 
method, since the accessibility of the following vehicle 
or the vehicle traveling on the neighboring traffic lane 
against the driver's own vehicle is detected automati- 
cally, it is expected that the information of the so- 
obtained accessibility to the driver (or passenger) 
allows to contribute the safety driving of the driver's own 
vehicle. 

[0005] In the above-mentioned method, however, 
there exists a problem to be solved. If an imaging envi- 
ronment around the vehicle changes due to climatic 
changes, for example, rainfall, snowfall, fog, etc., then 
the unclear pictures would be brought to cause the 
monitoring accuracy to be deteriorated. In the present 
circumstances, no measures has been taken against 
such a deterioration in monitoring accuracy of the 



above-mentioned monitoring method. In other words, 
there remains a possibility of providing the driver or pas- 
senger with an incorrect monitoring result including 
such an error. Consequently, according to the prior art 
5 monitoring method, it is impossible to provide the moni- 
toring result of the vehicle's circumference with high reli- 
ability. 

[0006] Under the circumstances, there has been 
eagerly expected a development of new technique 
10 which is capable of maintaining the high reliability in the 
monitoring result of the vehicle's circumference usually. 

SUMMARY OF THE INVENTION 

IS [0007] Under the circumstances, it is therefore an 
object of the present invention to provide a vehicle's cir- 
cumference monitoring method by which it is possible to 
always provide the monitoring result of the vehicle's cir- 
cumference with high reliability. 

20 [0008] It is another object of the present invention to 
provide a vehicle's circumference monitoring apparatus 
adopting such a monitoring method. 
[0009] The former object of the present invention 
described above can be accomplished by a vehicle's cir- 

25 cumference monitoring method of nwnitoring whether 
there is an obstacle in the neighborhood of a vehicle by 
applying a designated image processing on a picture 
about the neighborhood of the vehicle, the picture being 
brought by using an imaging unit installed on the vehi- 

30 de, the monitoring method comprising the steps of: 

judging whether the picture taken by using the 

imaging unit is improper or not; and 

when it is judged that the picture is improper, 

35 informing a passenger on the vehicle of a fact that 
the present monitoring about the vehicle's neigh- 
borhood is being deteriorated in its accuracy, on the 
assumption that the monitoring result on the ground 
of such an improper picture has a reduced reliability 

40 in accuracy. 

[0010] According to the monitoring method of the 
present invention, when the circumference monitoring 
accuracy is reduced, such a fact is informed to the pas- 

45 senger. Thus, it is possible to accomplish the high relia- 
bility in the monitoring results brought by this method. 
[001 1 ] According to the second aspect of the inven- 
tion, when it is judged in the judging step that there is a 
change from the proper picture into the improper pic- 

50 ture, it is immediately executed to inform the passenger 
of a fact that the present monitoring about the vehicle's 
neighborhood is being deteriorated in its accuracy, on 
the assumption that the monitoring result on the ground 
of such an improper picture has a reduced reliability in 

55 accuracy. 

[0012] Also in the above case, when the circumfer- 
ence monitoring accuracy is reduced, such a fact is 
informed to the passenger at once. Tlius, it is possible 
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to acxx>mpli8h the high reliability in the monitoring 
results. 

[0013] According to the third aspect of the inven- 
tion, when it is judged in the judging step that there is a 
change from the improper picture into the proper pic- 
ture, it is immediately executed to inform the passenger 
of a fact that the monitoring accuracy about the vehicle's 
circumference is recovered to a normal condition, on the 
assumption that the monitoring result on the ground of 
the proper picture has a recovered reliability in accu- 
racy 

[0014] In the above case, when the circumference 
monitoring accuracy is recovered to the normal condi- 
tion, then such a tact is informed to the passenger at 
once. Thus, it is possible to accomplish the high reliabil- 
ity in the monitoring results usually 
[0015] According to the fourth aspect of the inven- 
tion, the above judging step of the present monitoring 
method is canried out by applying a designated image 
processing on the picture taken by the imaging unit. 
[0016] Then, it is possible to realize the judgement 
of improper picture on the ground of the taken pictures 
with high accuracy 

[001 7] According to the fifth aspect of the invention, 
the judging step of the present monitoring method is 
carried out with reference to an actual Imaging environ- 
ment In the neighborhood of the vehicle. 
[0018] Therefore, it is possible to realize the judge- 
ment of irrproper picture white referring to the present 
Imaging environment including various weather, for 
example, rainfall, snowfall, fog. etc. 
[0019] The latter object of the present invention 
described above can be accomplished by a vehicle's cir- 
cumference monitoring apparatus adopting a method of 
monitoring whether there is an obstacle in the neighbor- 
hood of a vehicle by applying a designated image 
processing on a picture about the neighborhood of the 
vehicle, the picture being taken by using an imaging unit 
installed in the vehicle, the monitoring apparatus com- 
prising: 

an Improper picture judging unit forjudging whether 
the picture taken \3y using the imaging unit is 
improper or not; 

an informing unit for informing an outside of the 
apparatus that the monitoring accuracy about the 
vehicle's circumference Is deteriorated; and 
a drive control unit tor controlling the driving of the 
Informing unit; 

wherein, when it is judged by the Improper picture 
judging unit that the picture taken by the imaging 
unit is improper, then the driving control unit oper- 
ates to drive the informing unit on the assumption 
that the monitoring result on the ground of such an 
improper picture has a reduced reliability in accu- 
racy 

[0020] According to the monitoring apparatus of the 



present invention, when the circumference monitoring 
accuracy is reduced, such a feict is informed to the pas- 
senger. Thus, it is possible to always accomplish the 
high reliability in the monitoring results brought by this 
5 method. 

[0021] According to the seventh aspect of the 
invention, the improper picture judging unit in the above 
apparatus carries out the judgement of improper picture 
by applying a designated image processing on the pic- 
10 ture taken by using the Imaging unit 

[0022] Then, it is possible to realize the judgement 
of improper picture on the ground of the taken pictures 
with high accuracy 

[0023] According to the eighth aspect of the inven- 
ts tion, the improper picture judging unit In the above 

apparatus canries out the judgement of irrproper picture 

with reference to an actual imaging environment in the 

neighborhood of the vehicle. 

[0024] Also in the above apparatus, it Is possible to 
20 realize the judgement of improper picture while referring 

to the actual imaging environment including various 

weather, for example, rainfall, snowfall, fog, etc. 

[0025] These and other objects and features of the 

present invention will become more fully apparent from 
25 the following description and appended claims taken in 

conjunction with the accompany drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 
30 [Q026] 

Rg. 1 is a functional block diagram of a vehicle's cir- 
cumference monitoring apparatus of the present 
invention; and 

35 Fig. 2 is a time chart for explanation of the operation 
of the vehicle's circumference monitoring apparatus 
of the present invention. 

DESCRIPTION OF THE PREFERRED EMBODIMENT 

40 

[0027] One embodiment of the present invention 
will be described with reference to the drawings. 
[0028] Fig. 1 is a functional block diagram of a vehi- 
cle's circumference monitoring apparatus of the present 
45 invention. In Fig. 1 , reference numeral 1 1 designates 
the vehicle's circumference monitoring apparatus of the 
Invention. Fig. 2 is a time chart for explanation of the 
operation of the vehicle's circumference monitoring 
apparatus 11. 

so [0029] With reference to Fig. 1, we now describe 
the schematic constitution of the vehicle's circumfer- 
ence monitoring apparatus 1 1 which adopts the vehi- 
cle's circumference monitoring method of the invention. 
[0030] As shown in the figure, the vehicle's clrcum- 

55 ference monitoring apparatus 1 1 includes a CCD cam- 
era 13 operating as the imaging unit installed in the 
vehicle, a variety of sensors 15. an Improper picture 
judging part 23 operating as the Improper picture judg- 
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ing unit, an ECU 17 operating as the drive control unit, 
an alarm unit 19 and a display unit 21 both of which has 
a function as the informing unit, and an alarm stop 
switch 25. Note. In the specification, the word ** switch" 
will be referred to -SW- for short, hereinafter. 5 
[0031] Normally, the vehicle is equipped with a plu- 
rality of CCD cameras in respective appropriate posi- 
tions, for example, the vehicle's front section, the rear 
section, the side portions, the front/side part, the 
rear/side part, etc. In accordance with the requirements, 
these CCD cameras may be arranged in order to take 
pictures of various obstacles, for example, neighboring 
vehicles existing in the vicinity of the front section, the 
rear section, the side portions, the front/side part and 
the rear/side part of the driver's own vehicle. Of course, 
the vehicle may be equipped with only one CCD camera 
for purpose of Imaging a picture of the obstacles exist- 
ing in a designated direction from the vehicle. Note, the 
shown apparatus 11 has a single CCD camera 13 for 
ease of explanation. 

[0032] The group of sensors 15 includes of a turn 
signal SW outputting the indicating direction of a turn- 
signal, a steering angle sensor for detecting the steering 
angle of the vehicle, a vehicle speed sensor for detect- 
ing the velocity of the vehicle, a small lamp SW for out- 
putting the information of ON/OFF state in a not-shown 
small lamp, a headlight SW for outputting the informa- 
tion of ON/OFF state in a not-shown headlight, a gear 
position SW for outputting the information about the 
present position of a shift lever, etc. 
[0033] The ECU 1 7 has the following functions of: 

recognizing the obstacle(s) existing around the 
vehicle as the result of the application of a desig- 
nated image processing on a picture taken by the 
CCD camera 13; 

monitoring a relative positional relationship 
between the driver's own vehicle and the obstacle 
recognized by the above recognizing function; 
judging the magnitude of a critical level from the rel- 
ative positional relationship between the driver's 
own vehicle and the obstacle obtained by the mon- 
itoring function; 

commanding the alarm unit 19 to raise an alarm 
signal when the critical level is large in comparison 
with a predetermined threshold level; 
judging whether the picture taken by using the CCD 
camera 13 Is regarded as an improper picture; and. 
when the result of judgement brought by the above- 
judging function represents the presence of the 
improper picture, 

driving the alarm unit 1 9 on the assumption that the 
monitoring accuracy of the circumference is deteri- 
orated due to the presence of the improper picture. 

[0034] The alarm unit 1 9 has three functions as fol- 
lows. That is, the first function is to raise the alarm sig- 
nal on receipt of the alarm command outputted from the 



ECU 1 7. The second function is to inform the passenger 
(driver) of a feet that the nnonitoring accuracy is 
reduced, on receipt of a drive control command output- 
ted from the ECU 17. The third function Is to inform the 
passenger of a fact that the monitoring accuracy is 
recovered to the normal condition, on receipt of the 
drive control command also outputted from the ECU 1 7. 
[0035] Similarly, on receipt of the drive control com- 
mand outputted from the ECU 17. the display unit 21 
has a function to selectively display one of two informa- 
tions: one is an information about a fact that the moni- 
toring accuracy of the vehicle's circumference is 
reduced; another is an information about a fact that the 
monitoring accuracy of the vehicle's circumference is 
recovered to the normal condition. 
[0036] According to the embodiment, when there is 
generated an information that the monitoring accuracy 
of the vehicle's circumference is reduced, the alarm 
stop SW 25 is manipulated by the passenger. That is, 
the alarm stop SW 25 is constituted to have a function 
to stop such an operation of the alarm unit 19 as inform- 
ing the passenger of the large critical level. 
[0037] We now describe the operation of the above- 
mentioned vehicle's circumference nfx>nltorlng appara- 
tus 1 1 with reference to Figs. 1 and 2. 
[0038] First of all, the ECU 1 7 does apply the desig- 
nated image processing on a picture which has been 
taken by the CCD camera 13 and sequentially inputted 
to the ECU 17. Further In the ECU 17, it is executed to 
judge whether or not the so-processed picture Is Inap- 
propriate for judging the critical level about the driver's 
own vehicle. 

[0039] For example, in the vicinity of the rear sec- 
tion of the vehicle, there are provided some CCD cam- 
eras 13 in order to take pictures exhibiting the vehicle's 
rear, left-side and right-side views. In this case, these 
pictures are subjected to the above designated image 
processing, so that it Is candied out to judge whether the 
pictures are Improper or not 
[0040] As to the judgement of improper/jproper pic- 
tures, the above-mentioned application of the desig- 
nated image processing may t>e replaced with another 
form where the judgement Is carried out while referring 
to "an imaging environment" around the vehicle. Here, 
the imaging environment around the vehicle means an 
imaging situation containing the climatic changes, such 
as rainfall, snowfall, fog, etc. 

[0041] As the result of the judgement of 
improper/t^roper pictures. If it is judged that the picture 
on image is proper, rt is carried out in the ECU 17 to 
apply the designated image processing on the picture 
taken by the CCD camera 13. For example, the desig- 
nated image processing is represented by extracting the 
neighboring vehicle's lighting sections (e.g. small-lamp, 
headlight, etc.) and the profile of obstacles, as charac- 
terized points. While monitoring the relative positional 
relationship between the obstacle on recognition and 
the vehicle, the ECU 17 canies out the following opera- 
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tion. 

[0042] That is, on the ground of the vehicle infbrnna- 
tion (e.g. lighting conditions of turn signals, steering 
angle, vehicle speed) brought from the group of sensors 
1 5. it is determined whether or not the driver's own vehi- 5 
cie is now changing its traveling lane (or turning to the 
right or left). For example, if it is judged that the driver's 
own vehicle is now changing its traveling lane, then the 
ECU 1 7 judges the magnitude of crKical level on a syn- 
thetic consideration of the relative positional relation- 
ship between the obstacle and the vehicle and the 
approaching speed therebetween. Thus, it is executed 
to judge whether there is produced a possibility of colli- 
sion or involving the obstacle. Note, for comparison with 
the obtained magnitude of critical level, the ECU 17 has 
a predetermined threshold level stored in a not-shown 
memory. 

[0043] As the result of comparing the present criti- 
cal level with the threshold level, when it is judged that 
the driver's own vehicle may be subjected to the traffic 
collision or involving, the ECU 17 commands the alarm 
unit 19 to raise the alarm signal, on the assumption of a 
large critical level. 

[0044] On receipt of the command, the alarm unit 
19 raises the alarm signal representing the crrtical level 
being enormous. In this way. it is possible to inform the 
passenger on the vehicle of the enormous aitical level 
timely Together with the information, it can be expected 
to effect the great contribution on the vehicle's safety 
driving while preventing the occurrence of accident, 
such as collision, Involving, or the like, in advance. 
[0045] When It is judged that the proper picture on 
image varies to the improper picture, it is regarded in 
the ECU 17 that the monitoring based on the improper 
picture has a reduced reliability. Therefore, as shown 
with (a), (b) of Fig. 2. the ECU 1 7 commands the alarm 
unit 19 and the display unit 21 to give the passenger the 
information about the monitoring accuracy being deteri- 
orated. Then, on receipt of the command, the alarm unit 
19 provides the passenger with a voice message, for 
example, "Monitoring Accuracy is Reduced. Push Alarm 
Stop SW.", or any warning sound, for example, a buzzer, 
a chime, or the like. Also, the display unit 21 provides 
the passenger with a display message, such as "Moni- 
toring Accuracy is Reduced. Push Alarm Stop SW. or 
the lighting of a warning lamp. In this way, the alarm 
stop SW 25 is turned to its skJe of "alarm stop" by the 
passenger. Consequently, the ECU 1 7 commands the 
alarm unit 19 to stop the warning operation, as shown 
with (c), (d) of Fig. 2. On receipt of the command, the 
alarm unit 19 stops the warning operation based on the 
judgement of critical level (inactivation of alarm system). 
[0046] On the contrary, if it is judged that the 
inproper picture on image varies to the proper picture, 
it Is regarded in the ECU 17 that the nnonitoring based 
on the proper picture has a recovered reliability. There- 
fore, as shown with (a), (b) of Fig. 2. the ECU 17 com- 
mands the alarm unit 19 and the display unit 21 to give 



the passenger the information about the monitoring 
accuracy being recovered to the normal condition. On 
receipt of the command, the alarm unit 19 provides the 
passenger with a voice message, such as "Monitoring 
Accuracy is Recovered. Release Alarm Stop SW.", or 
any sound, for example, a buzzer, a chime, or the like. 
Also, the display unit 21 provides the passenger with a 
display message, such as "Monitoring Accuracy is 
Recovered. Release Alarm Stop SW.", or the light-out of 
a warning lamp. Under such a situation, when the pas- 
senger manipulates the alarm stop SW 25 to the side of 
"alarm recover", then the ECU 17 transmits a command 
of recovering the alarm operation to the alarm unit 19 
(activation of alarm system), as shown with (c), (d) of 
Fig. 2. On receipt of the command, the alarm unit 19 
starts the warning operation as the result of the judge- 
ment of the critical level. 

[0047] It will be understood by those skilled in the 
art that the foregoing description is one of preferred 
enrt>odiments of the disclosed shield tenninal and the 
manufacturing method. Various changes and modifica- 
tions may be made to the present invention without 
departing from the spirit and scope of the invention. 
[0048] For example, we hereinabove describe the 
embodiment where the stop and recovery of the alarm 
operation is controlled by the passenger through the 
alarm stop SW 25. While, the same SW 25 may be elim- 
inated in the modification. Then, if it is judged that the 
proper picture is shifted to the improper one, the alarm 
operation would be automatically stopped. Conversely, 
if it is judged that the improper picture is recovered to 
the proper one. the alarm operation would be automati- 
cally recovered. TTien, the voice message might be, for 
example, "Warning is stopped as accuracy is reduced" 
in case of stopping the alarm operation, "Warning is 
recovered as accuracy is normal" In case of recovering 
the alarm operation, and so on. 

Claims 

1. A vehicle's circumference monitoring method of 
monitoring whether there is an obstacle in the 
neighborhood of a vehicle by applying a designated 
image processing on a picture about the neighbor- 
hood of the vehicle, the picture being brought by 
using an imaging unit on the vehicle, the monitoring 
method comprising the steps of: 

judging whether the picture taken by using the 
imaging unit is improper or not; and 
when it is judged that the picture is improper, 
informing a passenger on the vehicle of a fact 
that the present monitoring about the vehicle's 
neighborhood is being deteriorated in its accu- 
racy, on the assumption that the monitoring 
result on the ground of such an improper pic- 
ture has a reduced reliability in accuracy. 
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2. A vehicle's circumference monitoring method as 
claimed in Claim 1 , wherein, when It is judged in the 
judging step that there is a change from the proper 
picture into the improper picture, it is immediately 
executed to inform the passenger of a fact that the 
present monitoring about the vehicle's neighbor- 
hood is being deteriorated in its accuracy, on the 
assumption that the monitoring result on the ground 
of such an improper picture has a reduced reliability 
In accuracy. 

3. A vehicle's circumference monitoring method as 
claimed in Claim 1 . wherein, when it is judged in the 
judging step that there is a change from the 
improper picture into the proper picture, it is imme- 
diately executed to inform the passenger of a fact 
that the monitoring accuracy about the vehicle's cir- 
cumference is recovered to a normal condition, on 
the assumption that the monitoring result on the 
ground of the proper picture has a recovered relia- 
bility in accuracy. 

4. A vehicle's circumference monitoring method as 
claimed in Claim 1 , wherein the judging step is car- 
ried out by applying a designated image processing 
on the picture taken by the imaging unit. 

5. A vehicle's circumference monitoring method as 
claimed in Claim 1 , wherein the judging step is car- 
ried out with reference to an actual imaging envi- 
ronment in the neighborhood of the vehicle. 

6. A vehicle's circumference monitoring apparatus 
adopting a method of monitoring whether there is 
an obstacle in the neighborhood of a vehicle by 
applying a designated image processing on a pic- 
ture about the neighborhood of the vehicle, the pic- 
ture being brought by using an imaging unit of the 
vehicle, the monitoring apparatus comprising: 

an improper picture judging unit for judging 
whether the picture taken by using the imaging 
unit is improper or not; 

an informing unit for informing an outside of the 
apparatus that the monitoring accuracy of the 
vehicle's circumference is deteriorated; and 
a drive control unit for controlling the driving of 
the informing unit; wherein, when it is judged by 
the improper picture judging unit that the pic- 
ture taken by the imaging unit is improper, then 
the driving control unit operates to drive the 
informing unit on the assumption that the mon- 
itoring result on the ground of such an improper 
picture has a reduced reliability in accuracy 

7. A vehicle's circumference nrtonitoring apparatus as 
claimed in Claim 6, wherein the improper picture 
judging unit can-ies out the judgement of improper 



picture by applying a designated image processing 
on the picture taken by using the imaging unit. 

8. A vehicle's circumference monitoring apparatus as 
5 claimed in Claim 6. wherein the improper picture 
judging unit carries out the judgement of improper 
picture with reference to the present Imaging situa- 
tion in the neighborhood of the vehicle. 
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Abstract of EP1 032209 

A vehicle's circumference monitoring method is 
provided together with an apparatus adopting the 
method. In the apparatus, it is judged whether a 
picture taken by a CCD camera 1 3 is 
inappropriate for monitoring the vehicle's 
circumference. When it is judged that the picture 
is improper, it is regarded in the apparatus that 
the monitoring based on the improper picture has 
a reduced reliability in monitoring results. Then, 
the apparatus informs a passenger on the vehicle 
of a fact that the monitoring accuracy is 
deteriorated. 
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